1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)?
The group has all agreed that the flaw is the use of the word “they”. It is unclear what “they” is referring to in the question. The group was confused on whether the “they” is the professional development activities or the teachers. Also, the group agreed that since the questions only have two options, the child has a 50/50 chance guessing the answer. With true and false tests, teachers can not really tell if students studied for the tests.
Evidence: Commandment 2:
This is an example of Commandment 2, because it says how sentences with pronouns can be unclear to whom the pronoun refers to. This is also breaking Commandment 2. It is unclear whether the administrators of the teachers need to know the information.
Binary Choice Item # 2 - flaw(s)?
The group has agreed that the flaw in the question is that the vocabulary is far too complex for students. The student may not be able to answer the question because they would not understand what the words even mean. We all agreed that giving kids words that are too advanced for them is not a good choice. Students might become frustrated and give up. Vocabulary should be suitable for the age.
Evidence:
Students would probably no be able to understand material unless they had a dictionary with them.
Binary Choice Item # 3 - flaw(s)?
We all agreed that this is an example of Commandment 3. Some students may be confused by the words. For example, “much more” or “almost” may confuse the student. The person giving out the test should not want to trick the students. The focus should be on finding out what the student knows, and testing their knowledge, rather than tricking them.
Evidence:
This is an example of Commandment 3, because this true/false question has an absolute qualifier. Also, this question provides students with unintended clues.
Binary Choice Item # 4 - flaw(s)?
The group agreed that the phrase is an example of eliciting thoughtfulness. This is an example, because the question focuses on a single concept, which is what a teacher should do when creating a binary choice assessment. The goal is to get students to think about the test items and their responses, rather than thinking in a superficial way.
Evidence:
The goal is to elicit thoughtfulness. Even though the items do not need to be simplistic, they should be brief.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :Read the following statement and answer the statements 1-4 as True or False.
**
Paul and John were going to the movies after school on Friday. They were going to see "Toy Story 3" at 6:30pm. After the movie, Paul and John are going to go get ice cream and then they are going to stay the night at John's house and then watch movies.
Answer True or False
1) Paul and John are proper nouns.
2) After is a verb.
3) The last sentence is a run on sentence.
4)Ice cream is NOT a noun.
Answers
1)True 2)False 3)True 4)False
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? This is breaking commandment #3 and the flaw is answer A.
Evidence: This is because the answer A is much longer than the other ones and the commandment tells us if one answer is longer than the other, the student will choose the longest answer. This is because there is more information and the more information a child sees, the more they feel that answer is correct. This also breaks the item writing guidelines which states to maintain a similar length for both categories of items. In this case A is much longer than B, C, or D.
Multiple-Choice Item # 2 - flaw(s)?
This breaks commandment #3 and the flaw is answer A.
Evidence: The flaw in this example is letter A. The end of the questions says the word “an”, if the student notices this they will automatically choose A because it is the only answer that starts with a vowel. This is an example of commandment 3 because there is an unintended clue. This would not be good when evaluating a student because we would not know if the student knew the correct answer or knew to choose the answer that began with a vowel.
Multiple-Choice Item # 3 - flaw(s)?
This is a writing guide example and the flaw in the question is answer D.
Evidence: This is an item writing guide example. The fact that A and B are such similar answers the student may choose all of the above. Also the student may know that A and B are correct and be unsure about C, but they will choose D anyway because they know at least two of the answers are correct.
Multiple-Choice Item # 4 - flaw(s)? This fits into the guidelines for writing a multiple choice test and answer D is the flaw.
Evidence: Choice D is much longer than the answers associated with A, B, or C. The length of question D will lead the student to believe it is the correct answer because it contains so much information. As teachers we should aim to make all answers similar in length so we can truly assess a student’s knowledge.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”** On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Column A
Column B
1. Bias-Detection
A. Determining DIF indices
2. Item Difficulty
B. Computing K-R formula
3. Reliability
C. Evaluating curricular alignment
4. Score Interpretation
D. Deriving percentiles
5. Validity
E. Calculating p// - values
Matching Item # 1 - flaw(s)?
This matching item example actually has a couple flaws. The matching item's directions the directions are not clear and precise enough. The directions are wordy, and definitely could have been written more clearly. According to the powerpoint, homogeneity is an important attribute of well constructed matching items. This question is confusing. If worded better and the choices were clearer students would not have such a difficult time solving this problem. We also believe that the responses are not in any significant order, responses are not written alphabetically or chronologically. Finally, another flaw is that list a has the same number or parts than list b.
Evidence:
The directions for Item Writing Guidelines for Matching Items should be clear and concise. They should also not be too wordy. If the directions are not clear then students become unaware of what the directions are asking of them. This section of the powerpoint also states that all of the left hand entries should be made the premises and the right side should all be responses. In addition, all of the responses should be in some form of significant. Things should be put in a logical matter such as alphabetically or in chronological order and not just all jumbled up. The reason why things should be ordered in a logical matter is to avoid giving unintended clues. Also, this example has 5 premises and 5 responses. There should be more responses than premises. A few extra responses should be given because then the student can simply guess the answer or use process of elimination for the remaining matches. If there are the same number or responses and premises then the teacher will not get accurate data about the student. As the powerpoint states, a student who knows 80% of the matches could easily eliminate others and figure out the answers. According to the powerpoint, the example stated above has quite a few different faults.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)?
The group has all agreed that the flaw is the use of the word “they”. It is unclear what “they” is referring to in the question. The group was confused on whether the “they” is the professional development activities or the teachers. Also, the group agreed that since the questions only have two options, the child has a 50/50 chance guessing the answer. With true and false tests, teachers can not really tell if students studied for the tests.
Evidence: Commandment 2:
This is an example of Commandment 2, because it says how sentences with pronouns can be unclear to whom the pronoun refers to. This is also breaking Commandment 2. It is unclear whether the administrators of the teachers need to know the information.
Binary Choice Item # 2 - flaw(s)?
The group has agreed that the flaw in the question is that the vocabulary is far too complex for students. The student may not be able to answer the question because they would not understand what the words even mean. We all agreed that giving kids words that are too advanced for them is not a good choice. Students might become frustrated and give up. Vocabulary should be suitable for the age.
Evidence:
Students would probably no be able to understand material unless they had a dictionary with them.
Binary Choice Item # 3 - flaw(s)?
We all agreed that this is an example of Commandment 3. Some students may be confused by the words. For example, “much more” or “almost” may confuse the student. The person giving out the test should not want to trick the students. The focus should be on finding out what the student knows, and testing their knowledge, rather than tricking them.
Evidence:
This is an example of Commandment 3, because this true/false question has an absolute qualifier. Also, this question provides students with unintended clues.
Binary Choice Item # 4 - flaw(s)?
The group agreed that the phrase is an example of eliciting thoughtfulness. This is an example, because the question focuses on a single concept, which is what a teacher should do when creating a binary choice assessment. The goal is to get students to think about the test items and their responses, rather than thinking in a superficial way.
Evidence:
The goal is to elicit thoughtfulness. Even though the items do not need to be simplistic, they should be brief.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :Read the following statement and answer the statements 1-4 as True or False.
**
Answer True or False
1) Paul and John are proper nouns.
2) After is a verb.
3) The last sentence is a run on sentence.
4)Ice cream is NOT a noun.
Answers
1)True
2)False
3)True
4)False
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?
This is breaking commandment #3 and the flaw is answer A.
Evidence:
This is because the answer A is much longer than the other ones and the commandment tells us if one answer is longer than the other, the student will choose the longest answer. This is because there is more information and the more information a child sees, the more they feel that answer is correct. This also breaks the item writing guidelines which states to maintain a similar length for both categories of items. In this case A is much longer than B, C, or D.
Multiple-Choice Item # 2 - flaw(s)?
This breaks commandment #3 and the flaw is answer A.
Evidence:
The flaw in this example is letter A. The end of the questions says the word “an”, if the student notices this they will automatically choose A because it is the only answer that starts with a vowel. This is an example of commandment 3 because there is an unintended clue. This would not be good when evaluating a student because we would not know if the student knew the correct answer or knew to choose the answer that began with a vowel.
Multiple-Choice Item # 3 - flaw(s)?
This is a writing guide example and the flaw in the question is answer D.
Evidence:
This is an item writing guide example. The fact that A and B are such similar answers the student may choose all of the above. Also the student may know that A and B are correct and be unsure about C, but they will choose D anyway because they know at least two of the answers are correct.
Multiple-Choice Item # 4 - flaw(s)?
This fits into the guidelines for writing a multiple choice test and answer D is the flaw.
Evidence:
Choice D is much longer than the answers associated with A, B, or C. The length of question D will lead the student to believe it is the correct answer because it contains so much information. As teachers we should aim to make all answers similar in length so we can truly assess a student’s knowledge.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”**
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)?
This matching item example actually has a couple flaws. The matching item's directions the directions are not clear and precise enough. The directions are wordy, and definitely could have been written more clearly. According to the powerpoint, homogeneity is an important attribute of well constructed matching items. This question is confusing. If worded better and the choices were clearer students would not have such a difficult time solving this problem. We also believe that the responses are not in any significant order, responses are not written alphabetically or chronologically. Finally, another flaw is that list a has the same number or parts than list b.
Evidence:
The directions for Item Writing Guidelines for Matching Items should be clear and concise. They should also not be too wordy. If the directions are not clear then students become unaware of what the directions are asking of them. This section of the powerpoint also states that all of the left hand entries should be made the premises and the right side should all be responses. In addition, all of the responses should be in some form of significant. Things should be put in a logical matter such as alphabetically or in chronological order and not just all jumbled up. The reason why things should be ordered in a logical matter is to avoid giving unintended clues. Also, this example has 5 premises and 5 responses. There should be more responses than premises. A few extra responses should be given because then the student can simply guess the answer or use process of elimination for the remaining matches. If there are the same number or responses and premises then the teacher will not get accurate data about the student. As the powerpoint states, a student who knows 80% of the matches could easily eliminate others and figure out the answers. According to the powerpoint, the example stated above has quite a few different faults.